Analysis of GB electricity demand and twitter data
UK ‘demand flexibility service’ DFS experiments/trials/‘live’ events by NG-ESO and retailers such as @OctopusEnergy
Relevant media articles:
Questions:
Q1: Is there a noticeable ‘dip’ in demand? (Can we even tell?)
Q2: Do tweets tell us anything useful about the events?
Part of https://github.com/dataknut/savingSessions
[1] "Loading libraries..."data.table dplyr flextable ggplot2 hashTagR hms knitr
TRUE TRUE TRUE TRUE TRUE TRUE TRUE
kableExtra lubridate readr rtweet tidytext wordcloud
TRUE TRUE TRUE TRUE TRUE TRUE Use the half-hourly NG-ESO ‘generation mix’ data as a reasonable proxy for ‘demand’. This also includes a useful estimate of carbon intensity.
Should probably use the demand data instead as it includes embedded wind & solar.
Variables of interest:
Use rtweet to collect tweets matching:
#savingSession or #savingSessions (not case sensitive) - this seems to be the preferred Octopus hashtag#DemandFlexibilityService (not case sensitive) - as used by NG-ESO, appears less frequently usedWe keep these datasets distinct as not all Octopus DFS events were also wider NG-ESO ‘live’ events.
We do NOT store the tweets in the repo for both ethical and practical reasons…
Note also that we may not be collecting the complete dataset of hashtagged tweets due to the intricacies of the twitter API.
It is important to note that both datasets are therefore likely to be partial.
Figure 1 shows the cumulative number of #savingSession tweets collected and marks the relevant events. Labels are likely to overlap where events are close together.
Figure 2 shows the cumulative number of #DemandFlexibilityService tweets collected and marks the relevant events. Labels are likely to overlap where events are close together.
NB: data collection for this hashtag only started on 23rd January - it looks like twitter’s search API does not return tweets more than x days old. People were tweeting this hashtag earlier than January…
Q1: Demand reduction
Simple comparison of the ‘demand response session’ day half-hourly demand with the mean half-hourly demand for the previous n weekdays or weekend days as appropriate. Default n is 10.
All sorts of caveats apply:
Q2: Twitter sentiment analysis
Inspired by https://www.tidytextmining.com/sentiment.html (sentiment analysis) and https://towardsdatascience.com/create-a-word-cloud-with-r-bde3e7422e8a (word clouds)
Extract the tweets for the day of each DFS event and the day after.
Remove stop words (to, the, and, a, for, etc) & profanity etc from the tweets and sentiment them.
In each case we report the number of negative and positive codings (according to tidytext::get_sentiments("bing")) for the unique words. This will add up to the Number of unique words by sentiment. We then report the total frequency of words that are negative or positive (which will add up to the total number of words).
Remember the size of the (positive/negative) words in the word cloud is relative to the count of all (positive/negative) words in the wordcloud, not the total word count.
(Not all retailers may have taken part in this one)
According to https://energycentral.com/news/octopus-energy-provide-108mw-grid-flexibility-first-%E2%80%98saving-session%E2%80%99:
“Over 200,000 households reduced their energy demand by 108MW collectively, the same as a gas power station can generate in an hour. If replicated by all UK energy suppliers, this would be over 1GW.”
But https://www.current-news.co.uk/news/octopus-energy-customers-provide-108mw-of-flexibility-during-first-saving-session quotes Octopus as actually saying:
“if the program was scaled to all electric smart meter customers with all supplier[s] in Great Britain, it could create over 1GW of flexible energy load”
The following plot compares GB electricity demand during the DFS event with mean GB electricity demand over the last n similar days (i.e. weekdays or weekends as appropriate).
hms | Comparison mean | Saving session | diff | pc_diff |
|---|---|---|---|---|
17:00:00 | 41,153.5 | 42,925.0 | 1,771.5 | 4.3 |
17:30:00 | 41,174.0 | 42,736.0 | 1,562.0 | 3.8 |
Confounding factors:
Note that the comparison plot/table shows the difference between the DFS event day and the previous similar days. This does not show what the event day energy use would have been in the absence of the intervention - we’d need a randomised control on the day for that (or a sophisticated demand model as the counterfactual).
Octopus customers’ 108 MW was 0.252 % of mean GB generation (42.925 GW) over the hour of the session (see NG-ESO data above).
108 MW over 200,000 customers is 5.4^{-4} MW per household or, more sensibly, 0.54 kW per household. A pretty reasonable reduction.
But if 200,000 produced a 108 MW power reduction (0.54 kW per household ), the 1 GW estimate implies we only have 1,851,852 GB households with a smart meter. Really? Or did they mean all suppliers taking part in the scheme?
Anyway, the latest data has 14 million domestic eletricity smart meters as of Q1 2022 - 45% of all domestic meters.
So:
The trouble is… these are octopus customers self-selecting into a trial. You just have to read the tweets to see that there are households with installed batteries, pausable heat pumps in well-insulated homes and EV V2G. At what point will we all be similarly equipped?
And just to keep in mind the carbon context:
Turning to hashtags, we only have #savingSessions tweets for this event.
Number of tweets: 642
Positive words…
Joining, by = "word"
Joining, by = "word"
Negative words…
Joining, by = "word"
Joining, by = "word"
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 87 |
| positive | 104 |
| sentiment | freq |
|---|---|
| negative | 203 |
| positive | 349 |
(Not all retailers may have taken part in this one)
The same caveats about the comparisons apply here…
hms | Comparison mean | Saving session | diff | pc_diff |
|---|---|---|---|---|
17:30:00 | 42,352.6 | 41,355.0 | -997.6 | -2.4 |
18:00:00 | 42,027.6 | 40,784.0 | -1,243.6 | -3.0 |
And just to keep in mind the carbon context:
Turning to hashtags, we only have #savingSessions tweets for this event.
Number of tweets: 729
Positive words…
Joining, by = "word"
Joining, by = "word"
Negative words…
Joining, by = "word"
Joining, by = "word"
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 107 |
| positive | 106 |
| sentiment | freq |
|---|---|
| negative | 197 |
| positive | 410 |
(Not all retailers may have taken part in this one)
The same caveats about the comparisons apply here…
hms | Comparison mean | Saving session | diff | pc_diff |
|---|---|---|---|---|
17:30:00 | 42,851.5 | 41,488.0 | -1,363.5 | -3.2 |
18:00:00 | 42,469.4 | 41,011.0 | -1,458.4 | -3.4 |
And just to keep in mind the carbon context:
Turning to hashtags, we only have #savingSessions tweets for this event.
Number of tweets: 728
Positive words…
Joining, by = "word"
Joining, by = "word"
Negative words…
Joining, by = "word"
Joining, by = "word"
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 77 |
| positive | 103 |
| sentiment | freq |
|---|---|
| negative | 206 |
| positive | 490 |
(Not all retailers may have taken part in this one)
The same caveats about the comparisons apply here…
hms | Comparison mean | Saving session | diff | pc_diff |
|---|---|---|---|---|
17:00:00 | 42,915.4 | 42,065.0 | -850.4 | -2.0 |
17:30:00 | 42,810.5 | 42,194.0 | -616.5 | -1.4 |
And just to keep in mind the carbon context:
Turning to hashtags, we only have #savingSessions tweets for this event.
Number of tweets: 305
Positive words…
Joining, by = "word"
Joining, by = "word"
Negative words…
Joining, by = "word"
Joining, by = "word"
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 40 |
| positive | 62 |
| sentiment | freq |
|---|---|
| negative | 83 |
| positive | 189 |
(Not all retailers may have taken part in this one)
The same caveats about the comparisons apply here…
hms | Comparison mean | Saving session | diff | pc_diff |
|---|---|---|---|---|
17:00:00 | 43,741.5 | 47,821.0 | 4,079.5 | 9.3 |
17:30:00 | 43,798.5 | 48,470.0 | 4,671.5 | 10.7 |
And just to keep in mind the carbon context:
Turning to hashtags, we only have #savingSessions tweets for this event.
Number of tweets: 585
Positive words…
Joining, by = "word"
Joining, by = "word"
Negative words…
Joining, by = "word"
Joining, by = "word"
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 41 |
| positive | 73 |
| sentiment | freq |
|---|---|
| negative | 138 |
| positive | 362 |
(Not all retailers may have taken part in this one)
The same caveats about the comparisons apply here…
hms | Comparison mean | Saving session | diff | pc_diff |
|---|---|---|---|---|
09:00:00 | 40,592.7 | 45,823.0 | 5,230.3 | 12.9 |
09:30:00 | 41,228.4 | 46,739.0 | 5,510.6 | 13.4 |
And just to keep in mind the carbon context:
Turning to hashtags, we only have #savingSessions tweets for this event.
Number of tweets: 248
Positive words…
Joining, by = "word"
Joining, by = "word"
Negative words…
Joining, by = "word"
Joining, by = "word"
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 22 |
| positive | 40 |
| sentiment | freq |
|---|---|
| negative | 37 |
| positive | 85 |
This was a more generalised call for DFS:
The same caveats about the comparisons apply here…
hms | Comparison mean | Saving session | diff | pc_diff |
|---|---|---|---|---|
17:00:00 | 44,751.7 | 46,086.0 | 1,334.3 | 3.0 |
17:30:00 | 44,894.4 | 45,684.0 | 789.6 | 1.8 |
And just to keep in mind the carbon context:
Turning to hashtags, we have both #savingSessions and #demandFlexibilityService tweets for this event.
Number of tweets: 598
Positive words…
Joining, by = "word"
Joining, by = "word"
Negative words…
Joining, by = "word"
Joining, by = "word"
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 88 |
| positive | 90 |
| sentiment | freq |
|---|---|
| negative | 179 |
| positive | 320 |
Number of tweets: 167
Positive words…
Joining, by = "word"
Joining, by = "word"
Negative words…
Joining, by = "word"
Joining, by = "word"
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 23 |
| positive | 32 |
| sentiment | freq |
|---|---|
| negative | 68 |
| positive | 111 |
This was another generalised call for DFS:
The same caveats about the comparisons apply here…
Also this ecent followed very soon after the last one so this will affect the comparison ‘baseline’ (previous n days).
hms | Comparison mean | Saving session | diff | pc_diff |
|---|---|---|---|---|
16:30:00 | 44,077.8 | 44,975.0 | 897.2 | 2.0 |
17:00:00 | 44,965.3 | 45,275.0 | 309.7 | 0.7 |
17:30:00 | 45,027.0 | 44,814.0 | -213.0 | -0.5 |
And just to keep in mind the carbon context:
Turning to hashtags, we have both #savingSessions and #demandFlexibilityService tweets for this event.
Number of tweets: 502
Positive words…
Joining, by = "word"
Joining, by = "word"
Negative words…
Joining, by = "word"
Joining, by = "word"
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 82 |
| positive | 76 |
| sentiment | freq |
|---|---|
| negative | 160 |
| positive | 237 |
Number of tweets: 67
Positive words…
Joining, by = "word"
Joining, by = "word"
Negative words…
Joining, by = "word"
Joining, by = "word"
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 9 |
| positive | 19 |
| sentiment | freq |
|---|---|
| negative | 36 |
| positive | 60 |